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Using a sample of 232 million T (45) -* BB events collected with the BABAR detector at the PEP-II B 
Factory in 1999-2004, we study B~ -> D°K* (892)" decays where K*~ -> K° s ir~ and D° -» K--IT+, 
K'tt+tt , K-tt+tt+tt' (non-CP final states); K + K~ , (CP+ eigenstates); K° s ix , K° s <j> and 

KgUi (CP— eigenstates). We measure four observables that are sensitive to the angle 7 of the 
CKM unitarity triangle; the partial-rate charge asymmetries Acp± and the ratios of the B-decay 
branching fraction in CP± and non-CP decays 1Zcp±- 



Acp+ = -0.08 ± 0.19 (stat.)± 0.08 (syst.) 

Acp- = -0.26 ± 0.40 (stat.) ± 0.12 (syst.) 

TZ C p+ = 1.96 ±0.40 (stat.) ±0.11 (syst.) 

TIcp- = 0.65 ± 0.26 (stat.) ± 0.08 (syst.) 

PACS numbers: 13.25.Hw, 14.40.Nd 



The measurement of CP violation in B-meson de- 
cays offers a means to over-constrain the unitarity tri- 
angle. A theoretically clean determination of the an- 
gle l=aig(-V ud V* b /V cd V* b ) is provided by the B~ -> 
£)(*)o_K"(*)- decay channels in which the favored b — > cus 
and suppressed b — > ucs tree amplitudes interfere 0, • 
Results on the B~ — » D^*>°K~ decays have been pub- 
lished by the BABAR 0, H and BELLE @, col- 
laborations. In this paper, we present a study based 
on the interference between B~ — > D°K* (892) and 
B~ —>■ D°K*~ when both D° and D° decay to the same 
CP eigenstate (D CP ). Reference to a charge conjugate 
mode is implied throughout this paper unless otherwise 
stated. 

We follow 00 and define: 



n 



CP± 



T(B~ 



LJ CP± JX 



r(B H 



LJ CP± JX 



T(B- 



A, 



CP± 



T(B~ 



D°K*- 
D°cp ± K*- 



)+T(B^ 
-T(B+ 



-> D°K*+) 
LJ CP± JX 



) 



T(B- -> D° CP± K*-) ± T{B+ 



LJ CP± JX 



Both Acp and IZcp carry CP-violating information. Ne- 
glecting D°-D° mixing, they can be expressed as follows: 



TZcp± — 1 ± 2rs cos 5 cos 7 
± 2rg sin S sin 7 

Acp± = ■ 



B ) 



(1) 

(2) 



TZcp± 

where S is the CP-conserving strong phase difference 
between the B~ — > D°K*~ (suppressed)and B~ — > 
D°K*~ (favored) amplitudes, r B ^ 0.1-0.3 Q is the mag- 
nitude of their ratio and 7 is the CP- violating weak phase 
difference. A value close to 60° is favored for 7 when one 
combines all measurements related to the unitarity tri- 
angle ,.9j. It is useful to introduce also new variables, 

x = rs cos(5 ± 7), (3) 



which are better behaved (more Gaussian) in the region 
where rs is small. 

To search for B~ — ► D CP K*~ decays we use data col- 
lected with the BABAR detector [Hj at the PEP-II storage 
ring. The sample corresponds to an integrated luminos- 
ity of 211 ffcT 1 at the T(AS) resonance (232 million BB 
pairs) and 20.4 fb _1 at an energy 40 MeV below the 
peak. 

To reconstruct B~ — > D°K*~ decays, we select K*~ 
candidates in the K*~ — ► K®ir~, — ► mode and 

D° candidates in eight decay channels, D° — > K~tt + , 
K^tt+tt , R-TT+TT+n- (non-CP final states); K + K~, 
7T+7T- (CP± eigenstates); and K°n°, K% K° s oj (CP- 
eigenstates). We optimize our event selection to mini- 
mize the statistical error on the signal yield, determined 
for each channel using simulated signal and background 
events. Particle identification is required for all charged 
particles except for the pions from decays. 

K® candidates are formed from oppositely charged 
tracks assumed to be pions with a reconstructed invariant 
mass within 13 MeV/c 2 (four standard deviations) from 
the known K® mass ^lj : n^K ■ All K® candidates are 
refitted so that their invariant mass equals m K o (mass 
constraint). For those retained to build a D CP _ candi- 
date the tracks are also constrained to emerge from a 
single vertex (vertex constraint). For those retained to 
build a K*~ we further require their flight direction and 
length be consistent with a K® coming from the interac- 
tion point. The K® candidate flight path and momen- 
tum must make an acute angle and the flight length in 
the plane transverse to the beam direction must exceed 
its uncertainty by three standard deviations. K*~ candi- 
dates are formed from a and a charged particle with 
a vertex constraint. We select K*~ candidates which 
have an invariant mass within 75 MeV/c 2 of the known 
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value 11]. Finally, since the K*~ in B~ -> D° K*~ is 
polarized, we require |cos 9h\ > 0.35, where Oh is the an- 
gle in the K*~ rest frame between the daughter pion and 
the parent B momentum. The helicity distribution dis- 
criminates well between a P-meson decay and an event 
from the e + e~ — * qq (q € {u, d, s, c}) continuum, since 
the former is distributed as cos 2 Or and the latter is flat. 

Some decay modes of the D° contain a it . We com- 
bine pairs of photons to form a 7r° candidate with a total 
energy greater than 200 MeV and an invariant mass be- 
tween 115 and 150 MeV/c 2 . A mass-constrained fit is 
applied to the selected ir° candidates. Composite parti- 
cles included in the CP— modes are vertex constrained. 
Candidate cf> (u>) mesons are constructed from K + K~ 
(7T + 7r~7r°) particle combinations with the invariant mass 
required to be within 12 (20) MeV/c 2 or two standard 
deviations of the known values flU . Two further require- 
ments are made on the lo candidates. The magnitude of 
the cosine of the helicity angle between the D° momen- 
tum in the rest frame of the lo and the normal to the 
plane containing all three decay pions, must be greater 
than 0.25. The Dalitz angle [13, defined as the angle be- 
tween the momentum of one daughter pion in the lo rest 
frame and the direction of one of the other two pions in 
the rest frame of the two pions, must have a cosine with 
a magnitude less than 0.9. 

Except for the R® s tP final state, all D candidates are 
mass and vertex constrained. We select D° candidates 
with an invariant mass differing from the known mass [TH 
by less than 12 MeV/c 2 for all channels except K®tt° 
(30 MeV/c 2 ) and R°io (20 MeV/c 2 ). These limits are 
about twice the corresponding RMS mass resolutions. 

To suppress the background due to e + e~ — > qq reac- 
tions, we require |cos 9b\ < 0.9, where 9 b is defined as 
the angle between the B candidate momentum in the 
T(4S) rest frame and the beam axis. In qq background 
events the cos 9 b distribution is uniform, while for B- 
mesons it follows a sin 2 9b distribution. We also use 
global event shape variables to distinguish between qq 
continuum events, which have a two-jet-like topology in 
the Y(AS) rest frame, and BB events, which are more 
spherical. We require |cos 9t\ < 0.9 where 9t is the 
angle between the thrust axes of the B candidate and 
that of the rest of the event. We construct a linear dis- 
criminant from cos 9t and Legendre monomials 0] 
describing the energy flow in the rest of the event. 

We identify B candidates using two nearly indepen- 
dent kinematic variables: the beam-energy-substitutcd 
mass itles = \/ {s/2 + po • pb) 2 /Po — P% an d the en- 
ergy difference AE = E* B — y/s/2, where E and p are 
energy and momentum, the subscripts and B refer to 
the e + e~-beam system and the B candidate respectively; 
s is the square of the center-of-mass (CM) energy and 
the asterisk labels the CM frame. For signal modes, the 
jjjes distributions are all described by the same Gaussian 
function Q centered at the B mass with a 2.6 MeV/c 2 



resolution (for D° — > K® ir° the peak is slightly wider, 
2.7 MeV/c 2 ). The AE distributions are centered on zero 
for signal with a resolution of 11 to 13 MeV for all chan- 
nels except K® tt° for which the resolution is asymmetric 
and about 30 MeV. We define a signal region through 
the requirement \AE\ < 50(25) MeV for R° tt° (all other 
modes). 

A background for B~ -> D (w + n-)K*-(K°n-) is the 
decay B~ — > D° '(K®ir + ir~)ir~ which contains the same 
final state as the signal but has a branching fraction 600 
times larger. We therefore explicitly veto any selected 
B candidate containing a K®ir + it~ combination within 
25 MeV/c 2 of the D° mass. No background remains. 

In those events where we find more than one acceptable 
B candidate (less than 25% of selected events depending 
on the D° mode), we choose that with the smallest x 2 
formed from the differences of the measured and true D° 
and K*~ masses scaled by the mass spread which in- 
cludes the resolution and, for the K*~ , the natural width. 
Simulations show that no bias is introduced by this choice 
and the correct candidate is picked at least 82% of the 
time. 

According to simulation of signal events, the total re- 
construction efficiencies are: 13.1% and 14.2% for the 
CP+ modes D° -> K+K~ and tt+tt-; 5.5%, 10.0% 
and 2.4% for the CP- modes D° -► K° s ir° , K°<j> and 
R° s lo; 13.3%, 4.3% and 8.2% for the non-CP modes 
D° -> R-TT+, R-tt+tt and R-ir+ir+ir- . 

To study BB backgrounds we look in sideband regions 
away from the signal region in AE and m D a . We define 
a AE sideband in the interval —100 < AE < —60 and 
60 < AE < 200 MeV for all modes except D° -> R°w° 
for which the inside limit is ±95 rather than 60 MeV. 
The sideband region in m^o is defined by requiring that 
this quantity differs from the D mass peak by more 
than four standard deviations. It provides sensitivity to 
doubly-peaking background sources which mimic signal 
both in AE and ibes- This pollution comes from either 
charmed or charmless J3-meson decays that do not con- 
tain a true D°. As many of the possible contributions 
to this background are not well known, we attempt to 
measure its size by including the m^a sideband in the fit 
described below. 

An unbinned extended maximum likelihood fit to Toes 
distributions in the range 5.2 < m^s < 5.3 GeV/c 2 
is used to determine yields and CP-violating quantities 
Acp and IZcp- We use the same Gaussian function Q 
to describe the signal shape for all modes considered. 
The combinatorial background in the ities distribution 
is modeled with a threshold function ^5| A. Its shape is 
governed by one parameter £ that is left free in the fit. We 
fit simultaneously toes distributions of nine samples: the 
non-CP, CP+ and CP— samples for (i) the signal region, 
(ii) the rri£)o sideband and (in) the AE sideband. We fit 
three probability density functions (PDF) weighted by 
the unknown event yields. For the AE sideband, we use 
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A. For the m D o sideband (sb) we use a s h ■ A + & s b ■ G, 
where Q accounts for the doubly-peaking B decays. For 
the signal region PDF , we use a ■ A + b ■ Q ± c • Q, where 
b = iVpcak is scaled from 6 s b according to the ratio of 
the mjjQ signal-window to sideband widths and c is the 
number of B ± — > D°K* ± signal events. The non-CP 
mode sample, with relatively high statistics, helps con- 
strain the PDF shapes for the low statistics CP mode 
distributions. The AE sideband sample helps define the 
A background shape. 

Since the values of £ obtained for each data sample 
were found to be consistent with each other, albeit with 
large statistical uncertainties, we have constrained £ to 
have the same value for all data samples in the fit. The 
simulation shows that the use of the same Gaussian pa- 
rameters for all signal modes introduces only negligible 
systematic corrections. We assume that the B decays 
found in the m D o sideband have the same final states as 
the signal and we fit the same Gaussian to the doubly- 
peaking B background. 

The doubly-peaking ^-background is assumed to not 
violate CP and is therefore split equally between the B~ 
and B + sub-samples. This assumption is considered fur- 
ther when we discuss the systematic uncertainties. The 
fit results are shown graphically in Fig.^aud numerically 
in Table IJ 



TABLE I: Results from the fits. For each D° mode class, 
we give the event yield, the peaking background contribution 
^Vpeak, Acp and Hop- The uncertainties are statistical only. 





Yield 


-/Vpcak 


Acp 


Hcp 


non-CP 


489 ± 27 


20.1 ± 5.6 






CP+ 


37.6 ± 7.4 


4.1 ± 1.3 


-0.08 ±0.19 


1.96 ± 0.40 


CP- 


14.8 ± 5.9 


3.0 ±2.3 


-0.24 ±0.35 


0.73 ±0.29 
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The statistical significance of the CP+ and CP— yields 
are 6.8 and 2.7 standard deviations, respectively. The 
yields for each individual mode are 23.1 ± 5.1 (K + K~), 
17.4 ±5.0 (7T+7T-), 10.9 ±4.1 (K°tt°), 3.1 ± 3.2 (K°<j>) 
and 3.8 ±2.7 (K°u>). 

Although most systematic errors cancel for Acp, an 
asymmetry inherent to the detector or data process- 
ing may exist. After performing the analysis on a 
high statistics B~ — > D°ir~ sample (not applying the 
K*~ selection), the final sample shows an asymmetry of 
—0.019 ± 0.008. We assign a systematic uncertainty of 
±0.027. The second substantial systematic effect is a 
possible CP asymmetry in the peaking background. Al- 
though there is no physics reason that requires the peak- 
ing background to be asymmetric, it cannot be excluded. 
We note that if there were an asymmetry -4 poa k, a sys- 
tematic error on Acp would be given by -4 pca k x \ , where 
b is the contribution of the peaking background and c the 
signal yield. Assuming conservatively |vA pea k| < 0.5, we 



FIG. 1: Distributions of mES in the signal region for the 
non-CP modes in B ± decays (a), the CP± modes in B + (b) 
and B~ (c) decays and the CP— modes in B + (d) and B~ (e) 
decays. The dashed curve indicates the contribution from the 
doubly-peaking B-background estimated from a simultaneous 
fit to the D° sideband (not shown). 



obtain systematic errors of ±0.06 and ±0.10 on Acp+ 
and Acp- respectively. 

Since IZcp is a ratio of rates of processes with different 
final states of the D°, we must consider the uncertain- 
ties affecting the selection algorithms for the different D 
channels. This results in small correction factors which 
account for the difference between the actual detector 
response and the simulation model. The main effects 
stem from the approximate modeling of the tracking effi- 
ciency (1.2% per track), the K® reconstruction efficiency 
for CP- modes of the D° (2.0% per K°), the tt° re- 
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construction efficiency for the R® ir° channel (3%) and 
the efficiency and misidentification probabilities from the 
particle identification (2% per track). A substantial ef- 
fect is the uncertainty on the measured branching frac- 
tions Altogether, we obtain systematic uncertainties 
equal to ±0.11 and ±0.055 for TZcp+ and TZcp- respec- 
tively. 

Another systematic correction is applied to the CP- 
measurements which arises from a possible CP+ back- 
ground for the K®(f> and K^oj channels. In this case, the 
observed quantities, A c p s _ and T^p_ are corrected: 

Acp- = (1 + e)A°£_ - eA C p+; TZcp- = jff^, 

where e is the ratio of CP+ background to CP— signal. 
There is little information on this CP+ background. An 
investigation in BABAR of the D° -> R-R+R° Dalitz 
plot indicates that the dominant background for D — > 
R^4> comes from the decay ao(980) — > R + R~, at the 
level of (25 ± 1)% the size of the <f>R® signal. We have no 
information for the ujK® channel and assume (30 ±30)%. 
Adding the most frequent K®tt mode which does not 
suffer such a CP+ pollution, we estimate e = (13 ± 7)%. 
The systematic error associated with this correction is 
±0.01 and ±0.04 for Acp- and TZcp-, respectively. 

To account for the non-resonant K®ir~ pairs under the 
K*~ , we vary by 2ir all the strong phases in a conserva- 
tive model which incorporates S-wave R n pairs in both 
b —* cus and b — > ucs amplitudes. This background 
induces systematic variations of ±0.051 for Acp± and 
±0.035 for TZcp±- We add the systematic uncertainties 
in quadrature and quote the final results: 

A C p+ = -0.08 ± 0.19 (stat.)± 0.08 (syst.), 

A C p- = -0.26 ± 0.40 (stat.) ± 0.12 (syst.), 

K C p+ = 1.96 ±0.40 (stat.) ±0.11 (syst.), 

TZcp- = 0.65 ± 0.26 (stat.) ± 0.08 (syst.). 

These results can also be expressed in terms of x defined 
in equation |3 

x+ = 0.32 ±0.18 (stat.) ±0.07 (syst.), 
x~ = 0.33 ±0.16 (stat.) ±0.06 (syst.), 

where the CP+ pollution systematic effects increase x + 
and x~ by 0.022 ± 0.012 and 0.019 ± 0.010 respectively. 
From equation £Q) we find r B = 0.30 ± 0.25. 

In summary, we have studied the decays of charged 
B-mesons to a if*(892)~ and a _D°, where the latter is 
seen in final states of even and odd CP. We express the 
results with IZcp, Acp and a; ± . These quantities can be 
combined with other D^R W measurements to estimate 



Tb more precisely and improve our understanding of the 
angle 7. 
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